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ASSESSMENT  OF  HELMET-MOUNTED  HUD 
SYMBOLOGY  REFERENCING  SYSTEMS 


ABSTRACT 


The  TTCP  Technical  Panel  2  (TP2)  developed  a  HUD  symbology  set  using  a 
"mixed"  referencing  system  in  which  symbols  portraying  spatial  analogue  information 
are  aircraft  or  world  referenced,  whereas  non-spatial  symbols  are  head  referenced.  One 
potential  advantage  to  having  non-spatial  symbols  referenced  to  head  position  is  that 
critical  flight  and  power  information  can  be  made  available  to  the  pilot  even  when  the 
pilot's  gaze  is  directed  to  the  side  of  the  aircraft,  such  as  during  sidestep  maneuvers.  It  is 
not  clear,  however,  whether  pilots  can  effectively  use  a  mixed  referencing  system.  For 
example,  one  potential  problem  is  that  depending  on  moment-to-moment  positioning  of 
the  head,  one  or  more  of  the  head  referenced  symbols  may  overlap  with  the  aircraft 
referenced  symbols.  This  may  create  intolerable  perceptual/cognitive  confusion  and  high 
mental  workload. 

There  were  two  objectives  of  the  present  research.  The  first  objective  was  to 
contrast  the  effectiveness  of  the  TP2  mixed  frames  of  reference  against  an  aircraft  and  a 
fully  head-referenced  configuration.  To  do  this,  two  ADS33-type  tasks  were  used: 
formation  flight  and  sidestep.  The  formation  flight  results  showed  that  there  was 
undifferentiated  performance  across  the  aircraft,  mixed,  and  head  FORs.  The  sidestep 
results  showed  an  advantage  for  the  mixed  and  head  FORs  over  the  aircraft  FOR  in  two 
primary  performance  measures:  maintaining  heading  and  altitude.  The  superiority  of  the 
mixed  and  head  FORs  for  the  sidestep  maneuver  was  also  reflected  in  the  pilots'  ratings. 

A  second  objective  of  the  present  research  was  to  take  a  step  toward  establishing  a 
paradigm  to  assess  the  effects  of  space-based  and  object-based  attention  in  processing  of 
HUD  symbology.  To  do  this,  pilots'  ability  to  discriminate  targets  that  appeared  on  a 
HUD  was  assessed  across  the  mixed,  aircraft  and  head-referenced  configurations.  The 
results  of  this  experiment  were  clear  in  showing  that  the  processing  of  information  on 
HUD  displays  is  affected  by  space-based  attention.  Importantly,  the  spatial  attention 
effects  occurred  in  the  head-referenced  configuration,  but  not  in  the  aircraft  or  the  mixed 
FORs.  The  comparison  of  the  head  versus  aircraft  conditions  supports  the  notion  that 
referencing  the  HUD  to  head  movements  creates  a  near  (HUD)  domain  perceptual  layer 
that  is  distinct  from  the  far  domain  of  the  external  scene.  On  this  view,  the  effect  of 
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spatial  attention  occurred  with  the  head  FOR  because  attention  is  assigned  to  the  HUD 
layer,  in  a  manner  similar  to  the  placing  of  attention  on  perceptual  objects.  With  the 
aircraft  referenced  configuration,  the  HUD  (near-domain)  and  the  external  scene  (far 
domain)  are  less  likely  to  form  distinct  perceptual  layers. 


Keywords:  Heads-Up  Display  (HUD),  Helmet-Mounted  Display  (HMD),  Symbology, 
Frame  of  Reference,  ADS33,  Attention,  Aviation  Displays. 
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SECTION  ONE:  GENERAL  OVERVIEW 


1.1  INTRODUCTION 

Helmet  Mounted  Displays  (HMDs)  offer  several  potential  advantages  for  flying 
rotary  and  fixed  wing  aircraft,  including  protection  from  laser  dazzling  as  well  as  the 
capability  to  fly  in  degraded  visual  conditions.  In  HMDs  equipped  with  visually 
coupled  systems  (VCS),  a  direct  view  of  the  external  world  is  projected  onto  the  HMD 
along  with  symbology  representing  primary  flight  and  power  information. 

In  November  1998,  technical  experts  from  the  allied  nations  in  the  Technical  Co 
operation  Panel  2  (TTCP  TP2)  established  a  prospective  symbology  set  for  use  in  a 
helmet-mounted  display.  This  symbology  set  was  implemented  on  the  Air  Crew 
Demonstrator  (ACD)  at  BAE  Systems,  Canada  (formerly  Canadian  Maconi  Company) 
and  assessed  in  an  initial  study  (SOW  3773-3HC16,  DSTA  2,  December  1998). 

The  TP2  symbology  set  (see  Figure  1 )  was  developed  based  on  an  assumed 
Search  and  Rescue  (SAR)  mission  in  which  a  helicopter  equipped  with  a  VCS  HMD 
clears  cloud  at  500  feet  and  establishes  contact  with  a  crash  site  under  degraded  visual 
conditions.  The  helicopter  is  to  descend  to  the  rescue  site  along  a  prescribed  glide  path 
over  rough  terrain  at  which  point  a  hover  is  achieved.  The  implementation  of  the  TP2 
symbology  set  in  the  present  research  conforms  to  the  specifications  listed  in  Statement 
of  Work  (SOW)  #3773-3HC16  (DSTA  2):  details  concerning  the  description  and 
function  of  the  various  symbols  can  be  obtained  from  the  SOW. 

The  original  TP2  symbology  set  is  a  "mixed"  referencing  system  in  which 
symbols  portraying  spatial  analogue  information  are  aircraft  or  world  referenced, 
whereas  non-spatial  symbols  are  head  referenced.  One  potential  advantage  to  having 
non-spatial  symbols  referenced  to  head  position  is  that  critical  flight  and  power 
information  can  be  made  available  to  the  pilot  even  when  the  pilot's  gaze  is  directed  to 
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the  side  of  the  aircraft,  such  as  during  hover  and  sidestep  maneuvers.  It  is  not  clear, 
however,  whether  pilots  can  effectively  use  a  mixed  referencing  system.  For  example, 
one  potential  problem  is  that  depending  on  moment-to-moment  positioning  of  the  head, 
one  or  more  of  the  head  referenced  symbols  may  overlap  with  the  aircraft  referenced 
symbols.  This  may  create  intolerable  perceptual/cognitive  confusion  and  high  mental 
workload. 

The  initial  assessment  of  the  TP2  set  (SOW  #3773-3HC16  DSTA  2)  compared 
the  TP2  mixed  reference  configuration  to  two  other  symbology  referencing 
configurations:  "aircraft"  and  "repeater".  All  of  the  symbology  configurations  used  the 
same  TP2  symbols  shown  in  Figure  #1.  For  the  aircraft-referenced  set,  all  of  the 
symbology  was  referenced  to  the  front  and  center  of  the  aircraft,  and  was  presented  only 
in  the  forward  field  of  view.  For  the  repeater  set,  all  of  the  symbology  was  aircraft 
referenced,  plus  some  symbology  that  was  not  tied  to  aircraft  orientation  (Rad  Alt, 
airspeed,  and  torque)  was  repeated  90°  to  the  left  and  right  periphery. 


1  Aircraft  Reference  Symbol 

2  Horizon  Reference  Symbol 

3  Flight  Path  Predictor 

4  Velocity  Vector 

5  Acceleration  Cue 

6  Airspeed 

7  Relative  Wind  Velocity 

8  Relative  Wind  Direction 

9  Torque 

10  Radar  Altitude/VSI 

11  Heading  Tape 

12  Waypoint  Indicator 

13  Turn  Indicator 


Figure  1:  TP2  Symbology  Set 
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The  results  of  the  initial  assessment  of  the  TP2  symbology  showed  that  the 
mixed  set  resulted  in  performance  that  was  as  good  as,  and  at  times  superior,  to 
performance  with  a  fully  aircraft-referenced  set.  In  addition,  pilots  indicated  a 
preference  for  the  mixed  FOR  as  compared  to  an  aircraft  FOR. 

The  ability  to  effectively  use  the  mixed  FOR  can  be  explained  in  terms  of  object¬ 
layering:  through  coherent  motion,  the  symbologies  that  were  head-referenced  appear  to 
exist  and  move  in  a  unitary  perceptual  layer  that  is  distinct  from  the  aircraft-  and  world- 
referenced  symbols. 

Despite  the  relative  utility  of  the  TP2  set  (as  demonstrated  in  the  initial  study) 
there  are  numerous  outstanding  issues  that  must  be  addressed.  One  category  of  issues 
relates  to  the  static  and  dynamic  qualities  of  the  various  symbols  in  the  TP2  set.  Of 
particular  concern  is  that,  at  times,  pilots  became  disoriented  when  using  the  mixed  set. 
Some  of  the  disorientation  was  attributable  to  confusion  that  arose  between  the  heading 
tape  and  the  horizon  line  (as  part  of  the  AI).  A  modification  to  the  TP2  heading  tape 
was  warranted. 

A  second  important  issue  relates  to  FOR.  In  the  initial  study  examining  the  TP2 
set,  a  fully  head-referenced  system  was  not  assessed.  Given  that  there  are  currently 
several  HMD  systems  that  require  complete  head-referencing  of  symbology  (e.g., 

ANVIS  HUD  and  possibly  early  versions  of  ESVS),  it  is  germane  to  evaluate  this 
approach  to  referencing  flight  and  power  symbology. 

1.2  PRESENT  OBJECTIVES 

There  were  two  primary  objectives  of  the  present  research.  The  first  objective 
was  to  compare  three  frames  of  reference:  the  mixed  and  aircraft  references  used  in  the 
initial  assessment  and  a  fully  head-referenced.  The  referencing  for  these  three  frames  of 


4 


P515014.PDF  [Page:  14  of  73] 


reference  are  shown  in  Table  1.  The  second  objective  was  to  take  a  step  toward 
establishing  a  paradigm  to  assess  the  effects  of  object  layering  in  HUDs. 

Table  1.  Symbology  referencing  for  the  three  experimental  conditions 


Symbol 

Aircraft 

Mixed 

Head 

Torque 

aircraft 

head 

head 

Hover 

aircraft 

aircraft 

head 

Attitude 

aircraft 

aircraft 

head 

Rad  Alt 

aircraft 

head 

head 

IAS 

aircraft 

head 

head 

Rate  of  Turn 

aircraft 

aircraft 

head 

Wind  direction  and 
velocity 

aircraft 

aircraft 

head 

Velocity  Vector 

aircraft 

aircraft 

head 

Heading  Tape 

world 

world 

world 

Lubber  Line 

aircraft 

aircraft 

head 

Heading  Select  Bug 

world 

aircraft 

world 

aircraft 

world 

head 

1 .3  CHANGES  TO  THE  ORIGINAL  TP2  SYMBOLOGY  SET 

The  primary  cause  of  pilots'  sense  of  disorientation  when  using  the  original  TP2 
set  was  attributed  to  the  heading  tape  which  remained  in  the  pilots'  field  of  view  during 
all  head  movements.  To  correct  this,  the  heading  tape  was  changed  from  a  360°  wrap 
around  configuration  to  a  60°  aircraft  referenced  symbology.  No  other  changes  were 
made  to  the  TP2  symbology  set. 
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1.4  EXPERIMENT  OVERVIEW 

Each  pilot  was  tested  across  two  days.  On  Day  1,  pilots  performed  two  ADS33- 
type  tasks.  On  Day  2,  pilots  performed  a  task  designed  to  assess  attentional  allocation  to 
the  symbology  while  flying  a  multifaceted  mission. 


1.4.1  Day  One 

On  Day  l,.a  formation  flight  task  and  a  side-step  task  were  used  to  assess  the 
effectiveness  of  the  aircraft  vs.  mixed  vs.  head  symbology  referencing  systems.  These 
tasks  conform  to  the  SAR  mission  adopted  by  the  TP2  committee  for  the  initial 
development  of  the  TP2  symbology  set.  In  addition,  these  tasks  are  well  suited  for 
indexing  differences  across  the  referencing  sets  because  (a)  head  movements  are 
intrinsic  to  good  performance  on  these  tasks  and  (b)  flight  and  power  information  on  the 
HUD  is  likely  to  be  read  and  used  to  maximize  performance  on  these  tasks  when  the 
external  scene  is  degraded. 


1.4.2  Day  Two 

On  Day  2,  pilots  flew  missions  that  included  takeoff,  hover,  enroute,  low-level 
flight  and  recce  components.  While  flying  the  missions,  coloured  targets  were  presented 
at  the  HUD  symbology  level  and  the  pilots  were  required  to  perform  a  speeded 
discrimination  task.  The  target  discrimination  task  was  performed  while  flying  with  the 
aircraft  vs.  mixed  vs.  head  referencing  systems. 
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1.4.3  Participants 

Six  male  Canadian  Forces  helicopter  pilots  aged  36  to  43  years  volunteered  for 
this  study  (see  Table  2).  The  pilots  had  an  average  of  over  3,000  hours  experience 
flying  rotary  aircraft.  Most  pilots  had  some  simulator  experience.  Two  of  the  pilots  had 
little  to  no  NVG  experience.  Seven  pilots  were  originally  scheduled  to  participate  in  the 
study.  One  pilot  experienced  simulator  sickness  brought  on  by  use  of  the  HMD,  and  did 
not  complete  the  study. 


1.4.4  Testing  Facility 

The  study  was  conducted  in  the  Human  Factors  Engineering  (HFE)  Laboratory, 
BAE  Systems  in  Kanata,  Ontario,  Canada.  The  primary  simulator  hardware/software 
components  utilized  for  this  experiment  included:  (a)  a  Silicon  Graphics-based 
workstation  local  area  network,  (b)  a  single-pilot  physical  flight  structure  with  aircraft 
seating  and  low  fidelity  flight  control  systems  (cyclic,  collective,  rudder  pedals),  (c)  an 
N-Vision  immersive  HMD  providing  a  79  degree  diagonal  binocular  field  of  view  at 
VGA,  (d)  an  external  scene  utility  modified  for  the  HMD,  (e)  a  utility  to  generate  the 
TP2  symbology  set,  (f)  a  facility  to  generate  auditory  input  for  the  formation  flight 
secondary  task,  and  (g)  a  rotary  wing  flight  model  (see  below). 

1.4.5  Flight  Model 

Flight  dynamics  were  modeled  using  HELISIM,  a  high-fidelity,  6  degree  of 
freedom  simulation  environment.  The  flight  model  utilized  in  the  experiment  was  based 
on  performance  data  for  a  Bell  412  helicopter.  A  Stability  Augmentation  System  (SAS) 
algorithm  was  active  in  the  flight  dynamics  model  for  the  duration  of  the  experiment. 
This  algorithm  was  not  implemented  in  the  initial  TP2  experiment. 
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Table  2:  Summary  of  pilot  experience 


Ss 

Training 

Rotary  Hrs. 

Simulator  Experience 

Hrs 

G 

Glasses 

1 

•  HT406  Standards  Flight 
Commander  until.  Dec  99 

•  Instructor  Pilot  Aug.  96  - 
Dec  99 

•  OFTT  Instructor  Sep  96  - 
Dec  99 

•  1700  hrs  overall 

•  CH  139Jet  Ranger 

•  Sea  King 

•  Sea  King  OFTT 

0 

Yes 

2 

•  Military  Pilot  Wings 
Graduate  1981 

•  Instructor  Bell  206  (1300 
hrs) 

•  Instructor  Griffon  (800 
hrs) 

•  3600  hrs  overall 

•  CHI 36  Kiowa 

•  CH  139  Jet  Ranger 

•  CH  135  Twin  Huey 

•  CT1 34  Musketeer 

•  CT1 14  Tutor 

•  Chi 46  Griffon 

•  Bell  212/412 
Simulator 

•  Griffon  Simulator 

60 

Yes 

3 

•  Tactical  Helicopter 
(NOE)  Fly 

•  SAR;  AB  Initio  Flight 
Instructor 

•  Test  Pilot 

•  3600  hrs  overall 

•  Bel!  206;  212 

•  MBB  105 

•  HV-  11  (Labrador) 

•  UH  1 

•  H  46 

•  H  53 

40 

Yes 

(Instrument 

Flight) 

4 

•  Instructor  83-86  3CFFTS 
CT-134  Musketeer 

•  2300  hrs  overall 

•  CHI 36  Kiowa 

•  CHI 35  Twin  Huey 

•  CH  1 39  Jet  Ranger 

•  CH  146  Griffon 

•  Bell  206;  Bell  212 

•  Bell  412 

•  none 

:  30 

yes 

5 

•  CH  -135  OUT  TAC 
Instructor  Pilot  3yrs. 

•  CH-135  NVG  IP  5yrs. 

•  CH-146  NVG  SPEC  2yrs 

•  3500  hrs  overall 

•  CH  136  Kiowa 

•  CH  139  Jet  Ranger 

•  CH  135  Twin  Huey 

•  CH  146  Griffon 

•  —10  hrs  Griffon  & 

FSI 

•  CT  114  Tutor 

600 

No 

6 

•  CT  134  Musketeer 

•  CT  114  Tutor 

•  CH  139  Jet  Ranger 

•  CH  135  Twin  Huey 

•  Kingair 

•  3300  Overall 

•  CH  139  Jet  Ranger 

•  CH  135  Twin  Huey 

•  Enstrom  Single 

Engine  Piston 

•  C  90  King  Air 

•  UTLAS/DCIEM  Sea 
King  Trials 

•  C  17  Globemaster 

2 

No  ! 
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SECTION  TWO:  ADS-33  TASKS 

2.1  PROCEDURE 

The  pilots'  personal  history/experience  was  collected  at  the  beginning  of  the 
study  on  Day  1 .  This  was  followed  by  a  brief  overview  of  the  simulator  and  the  set  of 
TP2  symbols,  a  briefing  regarding  the  purpose  and  layout  of  the  study  and  instructions 
how  to  set  the  optics  for  the  HMD.  This  was  followed  by  a  practice  free-flight  period 
(minimum  of  15  minutes).  For  this  period  pilots  were  instructed  to  attempt  a  variety  of 
flight  tasks  with  the  goal  of  familiarizing  themselves  with  the  TP2  symbology  set,  the 
flight  simulator,  and  the  flight  model.  No  data  was  collected  for  this  free-flight  portion 
of  the  study. 

Day  1  was  divided  in  a  morning  practice  session  and  an  afternoon  experimental 
session.  In  both  the  morning  and  the  afternoon  sessions,  pilots  performed  the  formation 
flight  task  followed  by  the  sidestep  task  using  each  frame  of  reference  (FOR)  system 
(aircraft,  mixed,  head).  The  morning  practice  session  consisted  of  5  trials  of  formation 
flight  and  5  trials  of  sidestep  per  each  referencing  system.  The  afternoon  experimental 
session  consisted  of  10  trials  per  each  task  (formation  flight,  sidestep)  per  each  FOR 
(aircraft,  mixed,  head)  system. 

At  the  conclusion  of  each  of  the  tasks  in  the  experimental  session,  pilots 
completed  modified  Cooper-Harper  questionnaires  (MCH)  (Wierwille  &  Casali,  1983) 
regarding  the  workload  associated  with  using  the  symbology  for  the  formation  flight  and 
the  sidestep  tasks.  Also  ratings  were  obtained  regarding  the  use  of  the  symbology  to 
perform  the  tasks,  interference  of  the  symbology  with  the  perception  of  the  external 
scene,  and  rankings  of  FOR  preferences.  Following  completion  of  these  questionnaires, 
responses  to  open-ended  questions  were  solicited  and  additional  comments  were  noted. 
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2.1.1  Formation  Flight  Task 

The  formation  flight  consisted  of  a  series  of  discrete  2-minute  trials.  At  the 
beginning  of  each  trial,  the  following  flight  parameters  were  set:  aircraft  attitude  (0 
degrees  pitch  and  roll),  altitude  (200  feet),  airspeed  (80  knots),  and  heading  (00  North) 
as  indicated  with  the  "bug"  on  the  heading  tape.  Pilots  were  instructed  that  their  primary 
task  was  to  maintain  the  altitude,  airspeed  and  heading  parameters. 

A  secondary  task  was  introduced  to  induce  the  pilots  to  engage  in  head 
movements,  thereby  highlighting  the  differences  across  the  aircraft  vs.  mixed  vs.  head 
referencing  conditions.  To  this  end,  a  voice  prompt  indicating  "left"  or  "right"  was 
presented  via  speakers.  This  prompt  occurred  approximately  every  10  seconds 
(randomly  determined).  Pilots  were  to  turn  their  head  accordingly  (left  or  right)  to 
determine  the  colour  of  a  light  fixed  to  the  nose  of  an  aircraft  flying  in  formation  on 
either  side  of  their  aircraft.  Upon  acquiring  the  light,  the  pilots  were  instructed  to  make 
a  speeded  trigger  response  if  the  light  was  red  and  no  response  if  the  light  was  green 
(50/50  probability). 

2.1.2  Side-step  Task 

The  sidestep  task  consisted  of  a  series  of  discrete  45-second  trials.  For  each  trial, 
pilots  were  positioned  at  the  center  of  the  left  hover  pad,  performed  a  take-off  activity,  a 
brief  hover  and  lateral  transition  to  the  right,  and  landed  on  a  second  target  hover  pad 
located  to  their  immediate  right,  while  following  the  center  row  of  three  rows  of  pylons. 
Pilots  were  instructed  that  accuracy  and  not  speed  should  be  the  focus  of  the  task  and 
were  instructed  to  maintain  a  heading  of  due  north,  an  altitude  of  15  feet,  and  attempt  to 
follow  the  center  row  of  the  three  rows  of  pylons.  A  tone  cued  the  start  of  the  trial, 
followed  by  a  voice  prompt  indicating  15,  30,  40,  and  45  seconds.  The  helicopter  was 
returned  to  the  initial  starting  location  to  start  the  next  trial. 
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2.2  RESULTS 

2.2.1  Formation  flight 

The  pilots'  primary  requirements  for  this  task  were  to  maintain  initial  heading, 
altitude  and  airspeed  parameters  while  responding  to  the  auditory  ("left  vs.  right")  cues 
to  look  to  the  left  or  right  side  of  the  aircraft. 

7h  i.  '  .i.  l  TP?,  evaluation  (SOW  #3773-3HC16)  showed  that  in  the  formation 
flight  task,  pilots  were  slightly  better  at  minimizing  deviations  in  heading,  altitude  and 
airspeed  with  the  aircraft  FOR  than  with  the  mixed  FOR.  As  summarized  below,  these 
differences  were  not  found  in  the  present  experiment. 

2.2. 1.1  Heading 

The  average  deviation  in  heading  is  shown  in  Figure  2.  A  3(FOR:  aircraft,  head, 
mixed)  x  10(Trial:  1  to  10)  repeated  measures  ANOVA  showed  no  significant  effects. 
The  null  effect  of  FOR  on  deviations  in  heading  differs  from  the  first  TP2  experiment 
where  the  aircraft  FOR  showed  a  slight  advantage  relative  to  the  mixed  FOR. 

Figure  2 


Heading  -  Deviation 


head  aircraft  mixed 

Frame  of  Reference 
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2.2. 1.2  Altitude 

The  average  deviation  in  altitude  is  shown  in  Figure  3.  A  3(FOR:  aircraft,  head, 
mixed)  x  10(Trial:  1  to  10)  repeated  measures  ANOVA  showed  no  significant  effects. 
The  null  effect  of  FOR  on  deviations  in  altitude  differs  from  the  first  TP2  experiment 
where  the  aircraft  FOR  showed  an  advantage  relative  to  the  mixed  FOR. 


Figure  3 


Altitude  -  Deviation 


Frame  of  Reference 


2.2.1.3  Airspeed 

The  average  deviation  in  airspeed  is  shown  in  Figure  4.  A  3(FOR:  aircraft,  head, 
mixed)  x  10(Trial:  1  to  10)  repeated  measures  ANOVA  showed  no  significant  effects. 
The  null  effect  of  FOR  on  deviations  in  airspeed  differs  from  the  first  TP2  experiment 
where  the  aircraft  FOR  showed  a  slight  advantage  relative  to  the  mixed  FOR. 
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Figure  4 


Airspeed  -  Deviation 


head  aircraft  mixed 

Frame  of  Reference 


2.2. 1.4  Subjective  Measures 

2.2.1. 4.1  Modified  Cooper-Harper 

After  performing  the  formation  flight  trials,  pilots  provided  MCH  ratings  for  the 
three  referencing  systems.  As  shown  in  Figure  5,  the  average  MCH  rating  across  the 
three  FOR  was  3.8,  which  corresponds  to  the  categories  indicating  "fair,  some  mild 
deficiencies"  -  to  -  "minor,  but  annoying  deficiencies".  The  MCH  ratings  did  not  differ 
significantly  across  the  three  FORs. 


Figure  5 


Cooper-Harper  Rankings  -Formation  Flight 
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2.2. 1.4.2  Use  of  Symbology  for  Formation  Flight 

Pilots  rated  the  degree  to  which  the  symbology  aided  performance  when 
performing  the  formation  flight  task.  These  ratings  are  shown  in  Figure  6.  Paired  t-tests 
showed  that  the  ratings  of  the  FORs  did  not  differ  significantly  from  one  another. 


Figure  6 


Did  you  use  the  Symbology  to 
aid  performance  in  the 
formation  flight  task 


Reference  Set 


2.2.1. 4.3  Interference  With  External  Scene 

Figure  7  shows  pilots’  ratings  of  the  extent  to  which  the  symbology  interfered 
with  the  perception  of  the  external  scene  during  the  formation  flight  task.  Paired  t-tests 
showed  that  the  ratings  did  not  differ  across  the  three  FORs. 
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Figure  7 


Interference  Rating 
Formation  Flight 


Aircraft 


Symbology  Reference 


2.2. 1.4.4  Preference  Rankings  for  Formation  Flight 

Figure  8  shows  the  preference  rankings  associated  with  using  the  three  FORs 
the  formation  flight  task.  Paired  t-tests  did  not  show  any  significant  differences  in 
preference  across  the  three  F ORs  for  this  task. 

Figure  8 


Overall  Preference  Ranking  for 
Formation  Flight 
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2.2. 1.5  Summary  of  Formation  Flight  Results 

In  sum,  the  heading,  altitude,  and  airspeed  measures  suggest  that  pilots  were  able 
to  use  the  three  frames  of  references  with  equal  effectiveness  to  control  heading, 
altitude,  and  airspeed  in  the  formation  flight  task. 
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2.2.2  Sidestep  Task 

For  the  sidestep  task  the  helicopter  was  initially  positioned  at  the  center  of  the 
left  hover  pad.  Pilots  were  to  perform  a  take-off,  a  lateral  transition  (sidestep)  to  the 
right  at  an  altitude  of  1 5',  and  then  land  on  a  second  target  hover  pad  located  to  the  right. 
Pilots  were  to  follow  the  center  row  of  three  rows  of  pylons:  deviations  in  fore/aft  from 
the  center  row  were  measured.  A  heading  of  00  North  was  to  be  maintained  throughout 
the  sidestep. 

2.2.2. 1  Head  Position 

When  performing  a  sidestep  maneuver,  pilots  typically  look  in  the  direction  that 
they  are  heading.  In  the  present  situation  pilots  should  look  right.  Figure  9  shows  head 
positioning  across  the  10  trials  for  each  of  the  three  symbology  conditions.  For  data 
management  reasons,  only  the  head  position  data  from  only  one  pilot  (#6)  is  shown  in 
Figure  9:  the  data  for  this  pilot  is  representative  of  the  complete  sample. 

As  shown  in  each  panel  of  Figure  9,  the  pilot  started  trials  by  looking  toward  the 
front  of  the  aircraft  (y  value  of  0)  and  then  turned  his  head  to  the  right  (negative  y 
values).  As  the  right  hover  pad  was  approached,  pilots  looked  back  toward  the  front  of 
the  aircraft  as  they  began  to  execute  a  hover  and  landing  maneuver  and  at  times, 
executed  a  head  movement  to  the  left  (positive  y  values)  to  check  their  position  against 
the  sidestep  cones  after  landing. 

Of  particular  interest  is  that  in  the  mixed  and  the  head  referenced  conditions, 
(middle  and  bottom  panels  of  Figure  9)  pilots  maintained  a  rightward  gaze  throughout 
the  sidestep.  With  these  FORs,  pilots  only  returned  their  gaze  toward  the  front  of  the 
aircraft  as  they  were  at  the  right  hover  pad.  For  the  aircraft  FOR  (top  panel),  gaze  was 
returned  to  the  front  (y=0)  numerous  times  throughout  the  sidestep:  this  was  done  to 
acquire  the  altitude  and  torque  information.  To  the  extent  that  minimizing  head 
movements  during 
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sidestep  maneuvers  is  important,  the  mixed  and  the  head  FORs  clearly  offer  an 
advantage  over  the  aircraft  referenced  set. 

For  the  following  analyses,  the  beginning  of  the  sidestep  maneuver  was  defined 
as  the  point  when  the  helicopter  reached  the  right  edge  of  the  initial  (left)  hover  pad. 
The  conclusion  of  the  sidestep  maneuver  was  defined  as  the  point  when  the  helicopter 
reached  a  distance  of  2  times  the  center-side  distance  from  the  left  edge  of  the  right 
hover  pad.  The  data  analyzed  below  was  combined  across  all  6  pilots  and  across  the  1 0 
trials  per  each  symbology  condition. 

2.2.2.2  Fore/ Aft  Movement 

Average  deviations  in  fore/aft  movement  are  summarized  in  Figure  10.  A 
3(FOR)  x  1  Off  rial)  repeated  measures  ANOVA  did  not  show  any  significant  effects. 
The  main  effect  of  FOR  was  not  significant:  the  absence  of  a  FOR  effect  is  expected 
given  that,  regardless  of  FOR,  the  information  provided  by  the  symbology  set  does  not 
bear  upon  fore/aft  control  of  the  aircraft  in  a  sidestep  task. 

Figure  10 

Sideslip  -  Fore/Aft  Deviations 
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The  effect  of  Trial  approached  significance,  F(9,45)  =  1.85,  Mse  =  0.144,  p  = 
.085.  As  shown  in  Figure  1 1,  performance  in  minimizing  deviations  in  the  fore/aft 
movement  of  the  aircraft  generally  improved  across  trials. 

Figure  11 


Sideslip  -  Fore/Aft  Deviation 
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2.2.2.3  Altitude 

Average  deviations  from  a  target  altitude  of  15  ft.  are  summarized  in  Figure  12. 
A  3(FOR)  x  10(Trial)  repeated  measures  ANOVA  showed  significant  main  effects  of 
FOR,  F(2,  10)  =  5.48,  Mse  =  4.052,  p  <  .026  and  of  Trial,  F(9,  45)  =  4.92,  Mse  =  1.41,  p 
<  001.  As  shown  in  Figure  10,  pilots  had  more  difficulty  maintaining  a  15'  altitude  in 
the  aircraft  than  the  mixed  and  head  referenced  sets.  The  mixed  and  head  references 
resulted  in  similar  performance. 
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Figure  12 


Sideslip  -  Altitude  Deviation 


Frame  of  Reference 


The  main  effect  of  Trial  on  altitude  deviation  is  illustrated  in  Figure  13.  As  can 
be  seen  in  Figure  13,  pilots'  performance  in  minimizing  deviations  from  the  target 
altitude  of  15'  improved  across  trials. 


Figure  13 


Sideslip  -  Altitude  Deviation 
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2 .2.2.4  Heading 

Average  deviations  around  a  target  heading  of  00  degrees  are  shown  in  Figure 
14.  A  3(FOR)  x  lO(Trial)  repeated  measures  ANOVA  showed  a  significant  linear  trend 
for  FOR,  F(l,5)  =  24.54,  Mse  =  1 .66,  p  <  .005.  This  supports  the  notion  that  pilots  are 
better  able  to  maintain  the  aircraft's  heading  in  the  head-referenced  set  because  the 
heading  tape  is  in  the  field  of  view  while  looking  to  the  side  in  the  sidestep  task.  There 
was  also  a  significant  linear  trend  for  Trial:  as  shown  in  Figure  15,  pilots'  ability  to 
minimize  deviations  in  heading  improved  across  trials. 

Figure  14 


Sideslip  -  Heading  Deviation 


Figure  15 
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2.2.2. 5  Subjective  Measures 
2.2.2.5.1  Modified  Cooper-Harper 

After  performing  the  sidestep  trials,  pilots  provided  MCH  ratings  for  the  three 
referencing  systems.  The  MCH  ratings  for  the  sidestep  task  are  shown  in  Figure  16. 
The  average  MCH  rating  across  the  three  FOR  was  2.0,  which  corresponds  to  the 
categories  indicating  "fair,  some  mild  deficiencies"  -  to  -  "minor,  but  annoying 
deficiencies".  Paired  t-tests  showed  that  for  the  sidestep  task,  the  aircraft  reference  was 
rated  also  significantly  hard  to  use  than  both  the  mixed  reference,  t(5)  =  5.66,  p  <  .006, 
and  the  head  reference,  t(5)  =  3.92,  p  <  .018.  MCH  ratings  of  the  mixed  and  head 
referenced  sets  did  not  differ. 

Figure  16 

(Larger  numbers  refer  to  higher  workload) 
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2.2.2.5.2  Use  of  symbology 

Pilots  rated  the  degree  to  which  the  symbology  aided  performance  when 
performing  the  sidestep  task.  These  ratings  are  shown  in  Figure  17.  Paired  t-tests 
showed  that  the  symbology  aided  performance  less  in  the  aircraft  configuration  than  in 
the  mixed  configuration,  t(5)  =  3.1 1,  p  <  .027,  or  the  head-referenced  configuration,  t(5) 
=  2.94,  p  <  .033.  The  ratings  did  not  differ  across  the  mixed  versus  head-referenced 
configurations.  This  finding  shows  that,  even  though  the  symbologies  were  identical 
across  FORs,  the  way  in  which  the  symbologies  are  referenced  is  important. 

Figure  17 


Did  you  use  the  Symbology  set 
to  aid  performance  in  the 
sidestep  task 


Reference  System 
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2.2.2.5.3  Interference  with  perception  of  external  scene 

Pilots  rated  the  degree  to  which  the  symbology  interfered  with  the  perception  of 
the  external  scene.  As  shown  in  Figure  18,  these  ratings  did  not  differ  across  the  three 
FORs. 


Figure  18 


Interference  Rating 
Side  Step  Task 


Symbology  Reference 


2.2.2.5.4  Preference  Rankings 

Figure  19  shows  the  preference  rankings  associated  with  using  the  three  FORs  in 
the  sidestep  task.  Paired  t-tests  showed  that  for  sidestep,  the  mixed  FOR  was  preferred 
over  the  aircraft  reference,  t(5)  =  7.32,  p  <  .002  and  the  head  FOR  was  also  preferred 
over  the  aircraft  reference,  t(5)  =  10.25,  p  <  .001 .  The  pilots'  preferences  for  the  mixed 
FOR  and  the  head  FORs  did  not  differ. 
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^  Figure  19 

(Low  number  refers  to  highest  preference) 


Overall  Preference  Ranking  for  Side 
Step 


Symbology  Reference 


2.1.2.6  Summary  of  the  Results 

The  pilots'  subjective  ratings  are  clear  in  showing  that  for  the  sidestep  task,  the 
mixed  FOR  and  head  FOR  are  preferred  over  the  aircraft  FOR,  and  as  requiring  less 
mental  effort  to  use  than  the  aircraft  FOR.  The  superior  preference  and  MCH  rankings 
for  the  mixed  and  head  FORs  may,  in  part,  reflect  the  fact  that  fewer  head  movements 
are  required  to  perform  a  sidestep  maneuver  when  altitude  and  torque  information  is 
available  in  the  direction  that  the  helicopter  is  moving.  On  this  view,  pilots  indicated 
that  they  used  the  task-relevant  symbology  (altitude,  torque)  more  with  the  mixed  FOR 
and  head  FOR  than  the  aircraft  FOR. 

The  sidestep  performance  measures  show  that  pilots  were  better  able  to  control 
the  aircraft's  altitude  and  heading  with  the  mixed  and  head  FORs  as  compared  to  the 
aircraft  FOR.  Heading  was  especially  well  controlled  with  the  head  FOR,  for  which  the 
heading  tape  was  in  the  pilot's  field  of  view  throughout  the  sidestep  maneuver. 
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4-s  2.3  DISCUSSION 

The  ADS33  tests  showed  that  there  was  undifferentiated  performances  in  all 
aspects  of  the  formation  flight  task  across  the  aircraft,  mixed,  and  head  FORs.  There 
are  several  possible  reasons  why  differences  across  the  FORs  were  not  found  in  the 
formation  flight  task. 

First,  the  pilots  exhibited  high  and  stable  levels  of  performance  in  this  task. 
Indeed,  the  plots  in  this  experiment  performed  at  a  much  higher  and  stable  level  than 
those  in  the  initial  TP2  ADS-33  experiment.  This  high  level  of  performance  may  be 
attributed  to  the  extensive  practice  session  given  prior  to  the  experimental  trials  in  the 
present  experiment. 

Second,  as  noted  by  the  pilots,  tended  not  to  rely  heavily  on  the  symbology  to 
perform  the  formation  flight  task.  To  this  end,  a  more  difficult  formation  flight  task  in 
which  the  visual  referents  are  degraded  would  be  more  likely  to  challenge  the  pilots  and 
exercise  the  symbology. 

Third,  the  flight  model  in  the  initial  TP2  ADS-33  experiment  was  somewhat 
unstable,  thereby  making  the  task  of  flying  the  aircraft  generally  quite  difficult. 

Upgrades  to  the  flight  model,  and  in  particular,  the  addition  of  a  partial  SAS  algorithm 
have  made  the  aircraft  much  more  stable.  This  addition  has  diminished  the  workload  of 
the  pilots,  possibly  to  the  extent  that  the  HUD  symbology  is  less  necessary  to  perform 
the  formation  flight  task. 

The  sidestep  results  showed  an  advantage  for  the  mixed  and  head  FORs  over  the 
aircraft  FOR  in  maintaining  heading  and  altitude.  The  superiority  of  these  two  FORs  for 
the  sidestep  maneuver  was  also  reflected  in  the  pilots'  ratings. 

In  sum,  both  the  mixed  and  head  referenced  FORs  offer  clear  advantages  over  an 
aircraft  referenced  configuration.  The  advantage  of  these  FORs  will  depend  on  the  task 
that  the  pilot  is  required  to  perform.  It  is  of  particular  importance  to  note  that  the  mixed 
FOR  did  not  result  in  any  disadvantages  relative  to  the  other  two  FORs.  This  suggests 
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that  having  some  symbology  yoked  to  head  movements  while  others  are  referenced  to 
the  aircraft  is  a  plausible  configuration  for  HUD  and  HMD  design. 
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SECTION  THREE:  ATTENTON  AND  HUD 
REFERENCING 


3.1  INTRODUCTION 

A  growing  body  of  experimental  and  neuropsychological  research  supports  the 
conclusion  that  attention  is  referenced  to  perceptual  groups  or  objects  within  the  visual 
field.  This  is  known  as  the  object-based  attention  hypothesis.  The  object-based 
attention  hypothesis  provides  an  account  of  attentional  effects  in  both  static  displays  and 
in  situations  where  objects  must  be  tracked.  An  in-depth  summary  of  the  object-based 
attention  hypothesis,  and  the  application  of  this  hypothesis  to  HUDs,  can  be  obtained  in 
Report  PWGSC  File  No:  W771 1-9-7577/A). 

The  object-based  attention  hypothesis  has  implications  for  research  and 
development  of  HUDs  and  for  the  integration  of  HUDs  into  HMDs.  For  example,  based 
on  Gestalt  principles,  perceptual  groupings  of  HUD  symbology  will  be  formed  based  on 
common  motion,  colour,  proximity,  closure  and/or  figure-ground  separation.  Object- 
based  attention  may  underlie  difficulties  associated  with  pilots’  need  to  process  near 
(HUD)  and  far  (external  scene)  domain  information:  near  and  far  domains  differ  along 
one  or  more  of  the  Gestalt  grouping  principles.  An  object-based  attention  framework, 
and  a  corresponding  paradigm  for  assessing  object-based  attention  effects,  would  be 
useful  for  gaining  a  metric  on  near  versus  far  domain  attentional  capture  and  cognitive 
tunnelling. 

The  objective  of  this  experiment  was  to  take  a  step  toward  establishing  a 
paradigm  to  assess  the  effects  of  object  layering  in  HUDs.  To  do  this,  the  discrimination 
of  targets  that  appeared  on  a  HUD  was  assessed  across  three  HUD  referencing 
configurations  described  in  section  on  DAY  1  testing:  aircraft,  mixed,  and  head.  It  was 
assumed  that  the  head  and  the  mixed  referencing  configurations  facilitate  the  perceptual 
grouping  of  the  HUD  symbology  into  a  near  domain  that  is  distinct  from  the  far  domain 
of  the  external  scene.  These  two  domains  are  termed  layers.  It  should  be  more  efficient 
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to  allocate  attention  to  targets  that  are  presented  on  the  HUD  layer  in  the  mixed  and  head 
referenced  configurations  as  compared  to  when  the  symbology  is  referenced  solely  to 
the  aircraft.  This  should  be  reflected  in  faster  (and  more  accurate)  target  discrimination. 


3.2  METHOD 

3.2.1  Participants,  Facility  and  Flight  Environment 

Participants  were  the  same  six  male  Canadian  Forces  helicopter  pilots  who 
participated  in  the  Day  1  ADS33  testing.  The  same  testing  facility  and  flight  model 
were  used  as  in  the  Day  1  testing. 

The  virtual  environment  used  for  this  study  was  generated  with  a  Silicon 
Graphics  Octane  workstation,  rendered  in  VGA  with  1024  X  768  lines  of  resolution  and 
displayed  with  a  refresh  rate  of  30  Hz.  The  environment  consisted  of  a  centrally  located 
airport,  with  four  major  ‘sites  of  interest’  in  each  comer  of  the  square  shaped 
environment.  The  terrain  consisted  of  flat  plains  populated  with  virtual  trees 
approximately  25-75  feet  in  height,  with  continual  clear  visibility.  Each  ‘site  of 
interest’  consisted  of  virtual  buildings  designed  with  the  intent  to  provide  areas  for 
participants  to  explore  and  conduct  tactical  maneuvers  with  the  various  symbology 
references  in  a  realistic  environment.  The  various  sites  ranged  from  a  400’  castle,  an 
enclosed  monastery  with  a  courtyard  large  enough  to  permit  hover-landings,  a  small 
village  consisting  of  three  small  houses,  and  a  farmhouse. 

3.2.2  Procedure 

The  object-layering  hypothesis  was  tested  by  presenting  two  circles 
simultaneously  at  random  within  three  possible  configurations.  When  the  targets  were 
presented,  they  would  encircle  the  radalt,  torque,  or  airspeed  indicators.  For  any 
possible  target  configuration,  only  two  of  three  indicators  were  encircled.  Both  circles 
could  either  appear  as  mauve,  or  sky  blue.  Participant  responses  were  recorded  by 
pressing  one  of  two  possible  triggers  on  the  cyclic  control  stick. 
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The  facilitator  monitored  the  position  of  the  participants  aircraft  within  the 
environment  by  way  of  a  moving  map  display  that  presented  the  aircraft  current  heading, 
relative  flight  parameters,  and  the  aircraft’s  current  position  in  relation  to  the  target  UFO 
and  major  buildings  within  the  environment. 


3.2.3  The  Flight  Task 

Pilots  flew  through  a  morning  practice  session  and  an  afternoon  experimental 
session.  During  each  session,  pilots  flew  three  ten-minute  missions  for  each  of  the 
symbology  reference  sets.  The  order  that  participants  flew  each  symbology  set  was 
counterbalanced  across  subjects. 

The  morning  sessions  were  used  to  allow  the  participants  to  practice  flying  the 
various  missions,  and  practice  the  target  discrimination  task.  Subjects  were  instructed  to 
fly  through  the  environment  by  following  heading  and  flight-parameter  directions 
provided  aurally  by  the  facilitator.  Each  route  provided  the  opportunity  for  the 
participants  to  experience  using  each  symbology  set  while  performing  circuits,  recces, 
hovers,  hover  landings,  hover  transits  and  NAV  tasks.  In  order  to  ensure  that 
participants  continually  performed  off-axis  head  movements,  participants  were 
instructed  to  scan  and  report  the  location  of  a  light-blue  "UFO"  during  transits  between 
‘sites  of  interest.’  The  transit  phase  of  each  mission  provides  the  opportunity  for  pilots 
to  perform  off-axis  head  movements  under  relatively  low  workloads  by  searching  for  the 
UFO.  The  facilitator  controlled  the  appearance  of  the  UFO,  randomly  placing  the  object 
at  various  locations  around  the  aircraft.  When  subjects  were  within  2km  of  a  task  area, 
they  were  instructed  to  refrain  from  performing  the  UFO  search  and  proceed  onto  their 
assigned  task. 
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3.2.4  The  Object  Layer  Task 

In  order  to  test  the  object-layering  hypothesis,  participants  performed  an  target 
discrimination  task  while  flying  the  various  missions.  Participants  were  instructed  to 
monitor  their  3  primary  flight  symbologies  (airspeed,  torque,  and  radalt  indicators)  for 
the  simultaneous  appearance  of  two  circular  targets.  The  two  targets  simultaneously 
appeared  encircling  two  of  the  three  flight  displays.  For  each  appearance,  the  targets 
would  either  appear  as  the  same  colour  (both  mauve,  both  light  blue)  or  as  different 
colours  (one  mauve,  the  other  blue).  Participants  were  asked  to  press  the  main  trigger 
on  the  cyclic  if  the  targets  were  the  same  colour  and  a  thumb  trigger  if  they  were 
different  colours.  In  order  to  ensure  that  participants  did  not  solely  focus  on  the 
symbology,  they  were  instructed  to  perform  the  target  task  without  sacrificing  flight 
performance. 


3.2.5  Design  —  Target  Task 

The  purpose  of  the  target  detection  task  was  to  test  both  the  object  layering 
hypothesis  and  spaced-based  attentional  hypothesis,  to  test  the  effects  of  clutter  within  a 
symbology  set,  and  to  provide  further  evidence  supporting  the  efficacy  of  head-tracked 
reference  systems.  The  design  of  the  object  layer  test  was  a  3(Frame  of  Reference  - 
FOR:  aircraft,  mixed,  head)  x  3(Location  of  targets:  radalt/torque,  radalt/airspeed, 
airspeed/torque)  x  2(Type  of  target:  same,  different)  repeated-measures  design,  where 
each  subject  experienced  each  reference  set  and  all  possible  target  combinations. 

For  each  ten-minute  mission,  a  total  of  30  targets  were  randomly  presented 
throughout  the  duration  at  approximately  10s  intervals.  Out  of  30  targets,  15  targets 
appeared  as  the  same  colour  and  1 5  as  different  colours.  For  each  colour  block,  five  of 
each  possible  symbology  combination  was  presented  (i.e.  targets  appearing  around 
radalt/torque,  radalt/airspeed,  and  airspeed/torque).  Therefore,  for  each  symbology 
reference,  subjects  experienced  a  total  of  90  trials,  consisting  of  45  same-trials,  45 
different-trials,  and  1 5  trials  of  each  target/symbology  combination. 
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3.3  RESULTS 

3.3.1  Response  latencies 

Latencies  to  correct  responses  were  analyzed  in  a  3(FOR:  head,  aircraft,  mixed) 
x  2 (Type:  same,  different)  x  3(Location:  1,  2,  3)  ANOVA  with  repeated  measures  on  all 
factors.  The  source  table  for  this  analysis  is  shown  in  Table  3. 


Table  3:  ANOVA  Source  Table 
Latency  Data:  FOR  x  Type  x  Location 


Source 

SUM  OF 

df 

MEAN 

F 

Sig. 

SQUARES 

SQUARE 

FOR 

12816.380 

2 

6408.190 

.504 

.619 

Error(FOR) 

127246.837 

10 

12724.684 

TYPE 

226207.683 

1 

226207.683 

8.377 

.034 

Error(SMDIF) 

135015.343 

5 

27003.069 

LOCAT 

72964  319 

2 

36482.160 

4.403 

.043 

Error(LOCAT) 

82854.168 

10 

8285.417 

FOR  *  TYPE 

8004.672 

2 

4002.336 

.543 

.597 

Error(FOR*TYPE) 

73665.642 

10 

7366.564 

FOR  *  LOCAT 

21469.834 

4 

5367.459 

.381 

.820 

Error(FOR*LOCAT) 

281832.061 

20 

14091.603 

SMDIF  *  LOCAT 

72048.289 

2 

36024.144 

7.011 

.012 

Error(TYPE*LOCAT) 

51379.402 

10 

5137.940 

FOR  *  TYPE  *  LOCAT 

66973  576 

4 

16743.394 

1.330 

.293 

Error(FOR*TYPE*LOCAT) 

251734.495 

20 

12586.725 

Error(FOR*TYPE*LOCAT) 

251734.495 

20 

12586.725 

As  shown  in  Figure  20,  there  was  a  significant  main  effect  of  Type  where 
responses  were  faster  on  same  than  on  different  trials:  faster  responses  on  same  trials 
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than  on  different  trials  is  typical  in  the  literature.  This  result  shows  that  the  target 
detection  paradigm  in  the  present  experiment  was  sufficiently  sensitive  to  index 
characteristics  of  the  stimuli/task.  There  was,  however,  no  main  effect  of  FOR  and  FOR 
did  not  interact  significantly  with  any  of  the  other  variables.  Although  caution  must  be 
exercised  when  interpreting  null  effects,  the  null  effect  of  FOR  suggests  that  the 
attentional  demands  associated  with  discriminating  the  targets  was  not  differentially 
influenced  by  the  type  of  HUD  symbology  referencing  system. 

Figure  20 


Target  Response  Latencies 
Frame  of  Refenenoe  x  SameO'ffenent 


Frame  of  Ftefermc 


There  was  also  a  significant  main  effect  of  Location  (see  Table  3).  As  shown  in 
Figure  21,  and  accord  with  the  spatial  hypothesis,  responses  were  generally  faster  when 
the  targets  appeared  in  the  radalt/torque  combination,  as  compared  to  the  radalt/airspeed 
or  the  torque/airspeed  combinations.  This  fits  with  the  spatial  hypothesis  because  the 
radalt/torque  symbols  are  spatially  closer  together  than  are  the  radalt/airspeed  and  the 
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torque/airspeed  combinations.  The  impact  of  Location  was  isolated  to  the  same  trials,  as 
indicated  by  a  significant  interaction  between  Location  and  Type. 

Figure  21 

(bars  are  95%  confidence  intervals) 


Target  Response  Latencies 
Location  x  Same/Different 


□  same 

□  different 


Location 


3.3.2  Error  Analyses 

There  were  two  types  of  errors.  Errors  in  classification  where  same  targets  were 
classified  as  different  and  vice  versa.  Also,  there  were  misses,  where  no  response  to 
targets  was  elicited. 

3.3.2.1  Classification  Errors 

The  percent  classification  errors  were  analyzed  in  a  3(FOR:  head,  aircraft, 
mixed)  x  2(Type:  same,  different)  x  3(Location:  1,  2,  3)  ANOVA  with  repeated 
measures  on  all  factors.  The  ANOVA  source  table  for  this  analysis  is  shown  in  Table  4. 
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There  was  a  main  effect  of  Type,  where  fewer  errors  were  made  on  same  than  on 
different  trials  (20.44%  vs.  28.44%).  This  coincides  with  the  latency  data  and  is 
consistent  with  same/different  effects  found  in  the  literature.  The  main  effect  of  FOR 
showed  a  trend  toward  where  fewer  errors  were  made  in  the  head  than  the  other  two 
referencing  systems.  This  trend  is  shown  in  Figure  22.  This  was  qualified  with  a  near 
significant  FOR  x  Location  interaction.  The  primary  source  of  this  interaction  is  that 
pilots  made  substantially  fewer  target  classification  errors  in  the  combination  of  the  head 
FOR  with  the  targets  being  located  at  the  radalt  /torque  locations.  Indeed,  as  shown  in 
Figure  23,  the  error  rate  was  approximately  half  that  found  in  the  other  FOR  x  location 
combinations. 

The  superior  performance  in  discriminating  targets  presented  in  the 
altitude/torque  combination  while  using  the  head  FOR,  does  not  appear  to  be  due  to 
chance:  this  superior  performance  was  consistent  across  all  6  pilots.  Also,  this  finding 
generally  concurs  with  the  latency  data  where  an  advantage  in  probe  responses  was 
found  for  the  altitude/torque  combination,  thereby  ruling  out  a  speed/accuracy  trade-off. 


Table  4:  ANOVA  Source  Table 
Classification  Errors:  FOR  x  Type  x  Location 


Source 

Type  III  Sum  of 

df 

Mean  Square 

F 

Sig. 

Squares 

FOR 

542  256 

2 

271.128 

.929 

.427 

Error(FOR) 

2919.518 

10 

291.952 

LOCAT 

1036.585 

2 

518.293 

2.411 

.140 

Error(LOCAT) 

2149.315 

10 

214.932 

SAMDIF 

1728.545 

1 

1728.545 

8.021 

.037 

Error(SAMDIF) 

1077.452 

5 

215.490 

FOR  *  LOCAT 

979.185 

4 

244.796 

2.421 

.082 

Error(FOR*LOCAT) 

2022.649 

20 

101.132 

FOR  *  SAMDIF 

40  608 

2 

20.304 

.451 

.649 

Error(FOR*SAMDIF) 

450.417 

10 

45.042 

LOCAT  *  SAMDIF 

157.567 

2 

78.783 

.592 

.572 

LOCAT  *  SAMDIF 

157.567 

2 

78.783 

.592 

.572 

Error(LOCAT*SAMDIF) 

1331  162 

10 

133.116 

Error(LOCAT*SAMDIF) 

1331.162 

10 

133.116 

FOR  *  LOCAT  *  SAMDIF 

211.905 

4 

52.976 

1.475 

.247 

FOR  *  LOCAT  *  SAMDIF 

211.905 

4 

52.976 

1.475 

.247 

Error(FOR*LOCAT*SAMDIF) 

718.333 

20 

35.917 

Error(FOR*LOCAT*SAMDIF) 

718  333 

20 

35.917 
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Figure  22 


Percent  Errors  on  Probe  Task 


c  20 

o 

o  15 
<?  10 


aircraft  mixed 

Frame  of  Reference 


head 


Figure  23 

(AS  =  airspeed;  RA  =  radalt;  TQ  =  torque) 


c  20 

4> 


Percent  Error  on  Probe  Task 


.  aircraft , 
.  mixed 


AS  /  RA  AS  /  TQ 


RA/TQ 


Location  of  Probes 
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3. 3.2.2  Target  Misses 

The  percent  misses  (no  response  to  targets)  were  analyzed  in  a  3(FOR:  head, 
aircraft,  mixed)  x  3(Location:  1,  2,  3)  repeated  measures  ANOVA.  On  average  5.52% 
of  the  targets  were  missed  by  the  pilots.  There  were  no  significant  effects  of  FOR  or 
Location  on  percent  Misses. 

3.4  DISCUSSION 

The  purpose  of  this  experiment  was  to  take  a  step  toward  developing  a  paradigm 
to  assess  the  functioning  of  object-based  and  space-based  attention  in  HMDs  equipped 
with  HUDs.  This  research  links  directly  to  laboratory  research  conducted  under 
PWGSC  File  No:  W771 1-9-7577/A,  that  showed  a  role  for  both  object-  and  space-based 
attention  in  the  processing  of  dynamic  displays. 

The  results  of  this  experiment  are  clear  in  showing  that  the  processing  of 
information  on  HUD  displays  is  affected  by  space-based  attention.  In  particular,  it  takes 
longer  to  interrogate  targets  that  are  presented  on  the  HUD  in  spatially  distant  locations. 

An  important  finding  in  this  experiment  is  that  these  spatial  attention  effects 
occurred  in  the  head-referenced  configuration,  but  not  in  the  aircraft  or  the  mixed  FORs. 
The  comparison  of  the  head  versus  aircraft  conditions  is  interesting  because  it  supports 
the  notion  that  referencing  the  HUD  to  head  movements  creates  a  near-domain 
perceptual  layer  that  is  distinct  from  the  far-domain  of  the  external  scene.  On  this  view, 
the  effect  of  spatial  attention  occurred  with  the  head  FOR  because  attention  is  assigned 
to  the  layer,  in  a  manner  similar  to  the  placing  of  attention  on  perceptual  objects.  With 
the  aircraft  referenced  configuration,  the  HUD  (near-domain)  and  the  external  scene  (far 
domain)  are  less  likely  to  form  distinct  perceptual  layers. 

It  was  predicted  that  the  mixed  FOR  would  also  create  distinct  perceptual  layers, 
and  that  this  would  facilitate  target  discrimination.  This  hypothesis  was  not  supported. 
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a  possible  confound  inherent  to  the  mixed  FOR  may  have  precluded  the  possibility  of 
finding  the  expected  advantage  with  this  FOR.  In  particular,  there  would  be  trials 
(possibly  many  trials)  where  the  targets  were  presented  while  the  head-referenced 
symbologies  in  the  mixed  set  were  physically  overlapping  the  other  symbols  on  the 
HUD  thereby  making  target  discrimination  difficult:  there  was  no  way  to  control  for  this 
confound  in  the  present  experiment. 

Further  mining  of  the  present  data  set  needs  to  be  done.  The  missions  flown  by 
the  pilots  in  this  experiment  were  purposefully  multi-faceted  and  included  relatively  low 
workload  components  such  as  enroute  flight,  and  high  workload  components  such  as 
hover  and  recce.  It  would  be  useful  to  examine  the  attentional  effects  as  a  function  of 
the  type  of  task  the  pilot  was  performing  when  targets  were  presented. 

In  sum,  the  present  research  represents  an  important  step  toward  establishing  a 
framework  for  examining  and  modelling  the  role  of  object-  and  space-based  attention  in 
the  processing  of  information  in  dynamic  displays.  The  combination  of  simulator  and 
laboratory  research  on  these  issues  will  have  a  direct  impact  on  the  development  of 
HUD  and  HMD  systems,  such  as  those  proposed  by  the  TP2  panel. 
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Human  Factors  Consulting 


VCS  HMD  Extension  Experiment 
Package  for  Subject  7 


Forms  Package 
for  Subject  n 
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Human  Factors  Consulting 


VCS  HMD  Extension  Experiment 
Package  for  Subject  7 


Conduct  of  Experiment  -  Day  1 


Initial: 


* 


Gather  Personal  History 
Provide  ACD  Overview 
Review  Experimental  Protocol 
Review  Questionnaire 
Sign  Consent  Form 


Sessions: 

•  Describe  Session  Segments  -  Familiarization  Flight,  Formation  Flight,  and  Side  Step  Manoeuvres 

•  Describe  Symbology  Set  and  Reference  Systems 

•  Fly  all  three  references  systems  (Reference  Systems  2  and  3  are  conducted  as  per 
Reference  System  1) 

•  Fly  null  symbology  segments  for  Side  Step  manoeuvres 


Post  Flights: 

•  Fill  out  questionnaire  (Modified  Cooper-Harper,  then  Subjective) 

•  Ranking  sheet 

•  Debriefing/Observations 

•  Provide  Contact  Sheet 
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Human  Factors  Consulting 


VCS  HMD  Extension  Experiment 
Package  for  Subject  7 


Personal  History 


Name: 


Ai*e. 


Sex- 


Training  Background: 


Approximate  Hours  on  Rotary  Wing  Aircraft: 
Types  of  Helicopters  Flown:  _ 


Simulated  Flight  Experience: 


Approximate  Hours  NVG: _  Are  Glasses  Required; 

Other  Pertinent  Information:  _ 


Contact  Information: 
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Human  Factors  Consulting 


VC S  HMD  Extension  Experiment 
Package  for  Subject  7 


ACD  Overview 


developed  by  CMC  under  contract  to  DND 

Objective  provide  an  environment  for  the  evaluation  of  proposed  Operator  Machine  Interfaces 

low  fidelity  demonstration  tool,  comprised  of  generic  flight  controls,  low  fidelity  flight  models  for  a  small 
library  of  fixed  and  rotary  wing  aircraft,  high  resolution  instrument  panel  and  an  integrated  external  scene 

latter  development  included  the  integration  of  a  Helmet  Mounted  Display  (HMD)  and  head  tracking  unit  to 
provide  a  360  degree  scene 

Note:  It  is  not  intended  that  the  ACD  fulfil  the  role  of  a  simulator.  The  level  of  fidelity  is  sufficient  as  a 

Human  Factors  Engineering  testbed.  These  analyses  spawn  further  testing  in  high  fidelity  systems 
and  operational  aircraft  in  the  equipment  definition  and  implementation  phases. 
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Human  Factors  Consulting 


VCS  HMD  Extension  Experiment 
Package  for  Subject  7 


^  Experimental  Protocol 


•  General 

1 .  Follow  subject  order  listed 


Subject  No. 

First  Reference 

Second  Reference 

Third  Reference 

1 

Aircraft 

Mixed 

Head 

2 

Aircraft 

Head 

Mixed 

3 

Mixed 

Aircraft 

Head 

4 

Mixed 

Head 

Aircraft 

5 

Head 

Aircraft 

Mixed 

6 

Head 

Mixed 

Aircraft 

2.  State  general  purpose  of  experiment  to  subjects 

•  Our  long-term  goal  is  to  develop  symbology >  sets  that  can  be  effectively  integrated  into  HMDs .  In  the  present 
study ,  we  are  examining  3  different  reference  systems  for  a  symbology  set  that  has  been  proposed  in  support  of 
a  joint  Canadian,  American,  British  and  Australian  programme.  The  Reference  Systems  being  evaluated  within 
this  context  include: 

□  Aircraft  referenced *  The  HUD  symbology *  is  presented  in  the  centre  of  the •  forward  y field  of  view  (dash  mounted) 
and  is  fixed  to  the  aircraft  system 

□  Mixed  referenced :  A  combined  reference  system  was  established  in  support  of  this  experiment,  that  blends 
Aircraft  reference  systems  with  a  Head  reference  systems  As  such ,  the  HUD  components  will  move 
independent  of  one  another  in  accordance  with  the  subject  reference  specification. 

□  Head  referenced:  In  accordance  with  the  Aircraft  Reference  system,  the  primary  HUD  symbology » set  will  be 
fixed  to  the  centre  of  the  forward  field  of  view.  The  entire  HUD  symbology  set  will  move  in  conjunction  with  the 
pilots  head  motion,  remaining  centrally  fixated  at  all  times. 

□ 

•  This  study  is  of  limited  scope  -  that  is,  we  are  not  trying  to  answer  all  possible  questions  at  once.  Instead,  we 
are  focusing  on  the  impact  made  on  basic  flight  tasks  by  the  reference  system  used  to  display  HUD  symbology 
information. 

•  To  do  this,  we  will  have  you  fly  a  generic  rotary  wing  model  through  partially  degraded  conditions  (similar  to 
dusk/fog)  using  each  of  the  three  reference  systems.  For  each  reference  system  you  will  be  flying  for 
approximately  I  hour,  for  a  total  of  3  hours. 

•  Your  primary  task  throughout  the  experiment  is  to  maximize  your  performance  within  the  specific  flight  tasks  by 
achieving/maintaining  specific  altitude,  heading,  and  airspeed  parameters.  The  data  collected  and  the 
performance  scores  generated  reflect  timing,  heading,  altitude  and  airspeed  information  as  appropriate . 

•  There  are  two  flight  modes  within  each  session,  including  Formation  Flight,  and  Side  Step  manoeuvres  To 
make  the  Formation  Flight  task  challenging,  we  will  be  introducing  a  secondary  observation  task  requirement 

•  Each  Formation  Flight  period  will  consist  of  ten  2  minute  periods.  At  the  beginning  of  each  period,  aircraft 
attitude,  altitude  and  airspeed  will  be  reset  to  their  initial  settings.  This  reset  will  be  preceded  by  a  brief 
intemiption  of  the  external  scene,  approximately  3  seconds  The  initial  settings  for  aircraft  attitude  are  0 
degrees  in  pitch  and  roll,  altitude  is  200',  an  airspeed  of  80  knots,  following  a  heading  dictated  by  a  bug  on  the 
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Heading  Tape  Also,  during  the  Formation  Flight,  as  a  secondaiy  task,  you  will  be  asked  to  determine  the 
colour  of  a  lamp  affixed  to  the  nose  of  the  aircraft  flying  in  formation  with  your  aircraft  Data  collection 
includes  heading  and  altitude  deviations 

•  The  Side  Step  period  entails  the  performance  of  a  take-off  activity1,  followed  by  a  brief  hover  at  an  altitude  of 
20-30  \  and  a  Side  Step  manoeuvre  to  depart  from  the  hover  zone  with  the  nose  of  your  aircraft  at  due  north 
The  aircraft  will  proceed  at  an  altitude  of  25  ',  to  the  next  hover  location,  and  attain  a  25 '  hover  over  the  target 
zone .  Maintain  a  constant  heading ,  altitude  and  path  to  target  Once  a  stable  hover  is  attained ’  perform  a 
landing  as  quickly  as  possible  Data  collection  includes  heading ,  flight  path,  altitude  and  timing  information 

•  Finally ,  after  all  sessions  have  been  completed,  we  will  ask  you  to  fill  out  a  few  questionnaires. 

•  Do  you  have  any  questions? 


•  Session  #1 

•  .  Train  on  how  to  set  HMD 

•  Free  flight  period  (approx.  15-30  min.) 

□  It  is  important  that  you  become  familiar  with  both  the  simulator  and  with  the  symbology  set  that  you  will  be 
using  in  this  session.  Therefore,  we  would  like  you  to  engage  in  free  flight  for  a  minimum  of  5  minutes  (we  will 
stop  you  after  5  minutes  and  more  time  will  be  given  if  you  wish)  We  would  like  to  recommend  attempting  each 
of  the  specific  manoeuvres  during  this  free  flight  period. 

□  Although  the  experiment  is  run  in  a  visually  obscured  environment  of  combined  fog  and  dusk  lighting 
conditions,  this  familiarization  period  will  be  conducted  in  a  visually  unrestricted  scene  An  opportunity  to  fly 
in  the  obscured  environment  will  also  be  provided . 

□  We  are  not  recording  ANY  data  during  this  time  period  So,  please  use  this  time  to  perform  any  type  of  flight 
task  that  you  want  with  an  eye  toward  learning  how  to  obtain  and  use  attitude  information  from  the  symbology 
set  as  it  is  presented.  At  any  time  during  this  initial  free  flight  period  please  ask  questions  and  try  to  get 
comfortable  with  the  setup  and  the  symbology . 

Block  #  1 :  F ormation  F  light 

Block  #2:  SideStep 

Block  #3  (follow-on  testing):  We  will  ask  you  to  fly  the  Side  Step  manoeuvres  without  any  symbology  information 

whatsoever. 


Once  the  trial  starts,  pretend  that  we  are  not  here:  DO  NOT  STOP  TO  MAKE 
COMMENTS  UNTIL  THE  END  OF  THE  TRIAL. 


•  Questionnaires 

•  Remove  subject  from  the  ACD,  take  to  a  quiet  room  to  complete  questionnaires 
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Consent  Form 


Ij _ _  having  reached  the  age  of  majority,  agree  to  participate  in  the 

research  project  entitled  HUD  Symbolog)  Research  under  the  direction  of  Dr.  Chris  Herdman  conducted  at 
BAE  Systems  Canada.  I  acknowledge  that  my  participation  in  this  research  project  is  completely  voluntary. 

I  have  been  advised  and  understand  the  description  of  the  project,  including  its  purpose,  methods  of  research 
and  the  risks  associated  with  my  participation. 

I  understand  that  the  findings  of  the  study  may  be  published,  but  my  anonymity  in  material  arising  from  this 
study  will  be  maintained.  In  no  way  will  my  name  be  identified  or  attached  to  the  study. 

I  understand  that  my  participation  in  this  study  is  entirely  voluntary  and  that  I  may  decide  to  stop 
participating  at  any  time  without  any  consequences  to  my  career. 

Details  of  the  study  have  been  explained  to  me  and  my  questions  about  the  study  have  been  answered. 

I  may  obtain  additional  information  about  the  project  and  have  any  additional  questions  answered  by 
contacting  Dr.  Chris  Herdman. 


Name  of  Participant 


Name  of  Witness 


Signature  of  Participant 


Signature  of  Witness 


Date 


Date 
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Rating: 

Reference  System: 


Ability  to  obtain  and  use  information  from  the  symbology  to  support  Formation  Flight 

Aircraft 


Symbology 

Characteristics 


Demands  on  Pilot  Associated  with 
Formation  Flight  Task 


Rating 


Excellent,  highly  Information  can  be  easily  obtained  from  the 
desirable  symbology  and  used  without  mental  effort 


Good,  negligible  I  Information  can  be  obtained  from  the  symbology  I 


1  deficiencies 


and  used  with  very  little  mental  effort 


Information  can  be  obtained  from  the  symbology 
air,  some  mi  d  an(j  usecj  Wlt^  a  jow  ancj  acceptable  level  of  3 

deficiencies 


mental  effort 


/  the  symbology 
satisfactory,  withom 
improvements,  for 
performing  > 
V  this  task9  / 


NO  Deficiencies 
►  warrant 
improvements 


Minor,  but 

annoying 

deficiencies 

Moderate, 

objectionable 

deficiencies 

Very  objectionable 
but  tolerable 
deficiencies 


For  desired  performance,  moderate  levels 
of  mental  effort  are  required 

For  adequate  performance,  moderate  levels 
of  mental  effort  are  required. 

For  adequate  performance,  extensive  but 
manageable  levels  of  mental  effort  are  required. 


//  Is  the  N. 

information  in  theN^ 
symbology  obtainable 
and  useable  within 

\a  tolerable  pilot  / 
workload9  s 


Deficiencies 

require 

improvements 


w  .  ,  ~  For  adequate  performance,  extensive  and  nearly  n 

Major  defiaenc.es  unmanageabie  mental  effort  is  requ.red  '  > 

w  ~  For  adequate  performance,  cl ose-to-maximum  0 

Major  defic.enc.es  and  unmanageab|e  menta|  efFort  ls  requlred  » 


Major  deficiencies! 


For  adequate  performance,  maximum  and 
intolerable  mental  effort  is  required. 


//  Can  \ 

information  \ 
be  obtained  from  \ 
the  symbology  and  used 

\for  performing  S 
this  task9  / 


Improvements _ ^1  Symbology  cannot  be  used  for  the  performance 

mandatory  — WMajor  deficiencies  Qpthls  ta^ 


START 


PILOT  DECISIONS 
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Rating: 

Reference  System: 


Ability  to  obtain  and  use  information  from  the  symbology  to  support  Side  Step 

Aircraft 


Symbology 

Characteristics 


Demands  on  Pilot  Associated  with 
Side  Step  Task 


Excellent,  highly 
desirable 

Information  can  be  easih  obtained  from  the 
symbology  and  used  without  mental  effort 

Good,  negligible 

Information  can  be  obtained  from  the  symbology 

deficiencies 

and  used  with  very  little  mental  effort 

Fair,  some  mild 
deficiencies 

Information  can  be  obtained  from  the  symbology 
and  used  with  a  low  and  acceptable  level  of 
mental  effort 

Rating 


1 


'  the  symbology  \ 
satisfactory,  without 
improvements,  for 
performing  y 
\  this  task7  x 


NO  Deficiencies 
>  ■  ►  warrant 

improvements 


Minor,  but 

annoying 

deficiencies 

For  desired  performance,  moderate  levels 
of  mental  effort  are  required. 

4 

Moderate, 

objectionable 

deficiencies 

For  adequate  performance,  moderate  levels 
of  mental  effort  are  required. 

5 

Very  objectionable 
but  tolerable 
deficiencies 

For  adequate  performance,  extensive  but 
manageable  levels  of  mental  effort  are  required 

6 

/  Is  the  \ 

/  information  in  the  \ 
symbology  obtainable 
and  useable  within 

\ a  tolerable  pilot  S 
,  workload9  X 


NO  Deficiencies 

► - ►  require 

improvements 


, .  .  ,  ~  .  For  adequate  performance,  extensive  and  nearly 

Majordef.cenc.es  unmanaJeableFmental  effort  is  required  '  ' 

4I  ,  r-  ■  For  adequate  performance,  close-to-maximum  0 

Majordefic.enc.es  and  unmanageabie  mental  effort  is  required  » 


I  Major  deficiencies 


For  adequate  performance,  maximum  and 
intolerable  mental  effort  is  required 


/  Can  X 

/  information  \ 
r  be  obtained  from  \ 
the  symbology  and  used 

\for  performing  / 
this  task9  x 


Improvements _ y  .  Symboloe\  cannot  be  used  for  the  performance 

mandatory  rplajor  deficiencies  Qpthis  ta^T 


START 


PILOT  DECISIONS 
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Rating: 

Reference  System: 


Ability  to  obtain  and  use  information  from  the  symbology  to  support  Formation  Flight 

Mixed  References 


Symbology 

Characteristics 


Demands  on  Pilot  Associated  with 
Formation  Flight  Task 


Rating 


Excellent,  highly 
desirable 

Information  can  be  easily  obtained  from  the 
symbology  and  used  without  mental  effort 

Good,  negligible 

Information  can  be  obtained  from  the  symbology 

deficiencies 

and  used  with  very  little  mental  effort 

Fair,  some  mild 
deficiencies 

Information  can  be  obtained  from  the  symbology 
and  used  with  a  low  and  acceptable  level  of 
mental  effort 

r  the  symbology 
satisfactory,  without > 
improvements,  for 
performing  > 

v  this  task0  / 


Deficiencies 
►  warrant 
improvements 


Mtnor,  but 

annoying 

deficiencies 

For  desired  performance,  moderate  levels 
of  mental  effort  are  required 

4 

Moderate, 

objectionable 

deficiencies 

For  adequate  performance,  moderate  levels 
of  mental  effort  are  required. 

5 

Very  objectionable 
but  tolerable 
deficiencies 

For  adequate  performance,  extensive  but 
manageable  levels  of  mental  effort  are  required. 

6 

//  Is  the  \ 

information  in  theV 
symbology  obtainable 
and  useable  within 

\a  tolerable  pilot  / 
workload0  / 


Deficiencies 
►  ■  ►  require 

improvements 


Major  deficiencies 

For  adequate  performance,  extensive  and  nearly 
unmanageable  mental  effort  is  required. 

7 

Major  deficiencies 

For  adequate  performance,  close-to-maximum 
and  unmanageable  mental  effort  is  required 

8 

Major  deficiencies 

For  adequate  performance,  maximum  and 
intolerable  mental  effort  is  required. 

9 

//  Can  X 

information 
be  obtained  from  \ 
the  symbology  and  used 

\for  performing  y 
this  task?  / 


Improvements _  I  Symbology  cannot  be  used  for  the  performance 

mandatory  — WMajor  deficiencies  0fthista^ 


START 


PILOT  DECISIONS 
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Rating: 

Reference  System: 


Ability  to  obtain  and  use  information  from  the  symbology  to  support  Side  Step 

Mixed  References 


Symbology 

Characteristics 


Demands  on  Pilot  Associated  with 
Side  Step  Task 


Rating 


Excellent,  highly 
desirable 

Information  can  be  easily  obtained  from  the 
symbology  and  used  without  mental  effort 

Good,  negligible 

Information  can  be  obtained  from  the  symbology 

deficiencies 

and  used  with  very  little  mental  effort 

Fair,  some  mild 
deficiencies 

Information  can  be  obtained  from  the  symbology 
and  used  with  a  low  and  acceptable  level  of 
mental  effort 

/  Is  x 

r  the  symbology  X 
satisfactory,  without 
improvements,  for 
performing  / 
X  this  task9  / 


NO  Deficiencies 
*  •  ►  warrant 

improvements 


Minor,  but 
annoying 

deficiencies  I 

Moderate, 

objectionable 

deficiencies 

Very  objectionable 
but  tolerable 
deficiencies 


For  desired  performance,  moderate  levels 
of  mental  effort  are  required 

For  adequate  performance,  moderate  levels 
of  mental  effort  are  required. 

For  adequate  performance,  extensive  but 
manageable  levels  of  mental  effort  are  required 


/  Is  the  X 
r  information  in  the  Ny 
symbology  obtainable 
and  useable  within 

\ a  tolerable  pilot  / 
.  workload9  / 


ND  Deficiencies 

> - ►  require 

improvements 


{Major  deficiencies] 


For  adequate  performance,  extensive  and  nearly 
unmanageable  mental  effort  is  required. 


...  ,  ~  .  For  adequate  performance,  close-to-maximum  0 

Major  defic.enc.es  &nd  unmanageable  mental  effort  is  required  » 

x ^  .  .  For  adequate  performance,  maximum  and  ~ 

Majordefic.enc.es  intolerabIe  mental  effort  is  required  9 


X  Can  X 

S  mtormation 
/  be  obtained  from  \ 
the  symbology  and  used 

\for  performing  / 
this  task9  / 


Improvements! 
mandatory  | 


x ,  ,  _  .  Symbologv  cannot  be  used  for  the  performance  ^  « 

Majordefic.enc.es  10 


START 
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Rating: 

Reference  System: 


Ability  to  obtain  and  use  information  from  the  symbology  to  support  Formation  Flight 

Head  Reference 


Symbology  Demands  on  Pilot  Associated  with 


Characteristics 


Formation  Flight  Task 


Rating 


Excellent,  highly 
desirable 

Information  can  be  easily  obtained  from  the 
symbology  and  used  without  mental  effort 

Good,  negligible 

Information  can  be  obtained  from  the  symbology 

deficiencies 

and  used  with  very  little  mental  effort 

Fair,  some  mild 
deficiencies 

Information  can  be  obtained  from  the  symbology 
and  used  with  a  low  and  acceptable  level  of 
mental  effort 

f  the  symbology  \ 
satisfactory,  without* 
improvements,  for 
performing  > 
\  this  task0  / 


Deficiencies 

> - ►  warrant 

improvements 


Minor,  but 

annoying 

deficiencies 

Moderate, 

objectionable 

deficiencies 

Very  objectionable 
but  tolerable 
deficiencies 


For  desired  performance,  moderate  levels 
of  mental  effort  are  required. 

For  adequate  performance,  moderate  levels 
of  mental  effort  are  required 


For  adequate  performance,  extensive  but 
manageable  levels  of  mental  effort  are  required 


/  Is  the  \ 
/information  in  the  N. 
symbology  obtainable 
and  useable  within 

\a  tolerable  pilot  / 
.  workload?  / 


Deficiencies 

require 

improvements 


w  ,  r-  •  For  adequate  performance,  extensive  and  nearly  n 

Major  defic.enc.es  unmanageable  mental  effort  is  requlred.  7 

v .  .  ,  ~  .  .  For  adequate  performance,  close-to-maximum  q 

ajor  e  iciencies  an(j  unmanageable  mental  effort  is  required  ® 

* ,  ,  For  adequate  performance,  maximum  and  n 

Major  def, cenc.es  lnto,erag,e  mentai  effort  .s  requ, red.  9 


//  Can  \ 

information  \ 
be  obtained  from  \ 
the  symbology  and  used 

\for  performing  / 
tins  task9  / 


Improvements 

mandatory 


. ,  ,  ~  Symbology  cannot  be  used  for  the  performance  -g  ^ 

Majordefic.enc.es  Jthista^  10 


START 
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Rating: 

Reference  System: 


Ability  to  obtain  and  use  information  from  the  symbology  to  support  Side  Step 

Head  Reference 


Symbology 

Characteristics 


Demands  on  Pilot  Associated  with 
Side  Step  Task 


Excellent,  highly 
desirable 

Information  can  be  easily  obtained  from  the 
symbology  and  used  without  mental  effort 

Good,  negligible 
deficiencies 

Information  can  be  obtained  from  the  symbology 
and  used  with  very'  little  mental  effort 

Fair,  some  mild 
deficiencies 

Information  can  be  obtained  from  the  svmbology 
and  used  with  alow  and  acceptable  level  of 
mental  effort 

Rating 


1 


'  the  symbology 
satisfactory,  without 
improvements,  for 
performing  7 
X  this  task9  / 


NO  Deficiencies 

> - ►  warrant 

improvements 


Minor,  but 

annoying 

deficiencies 

Moderate, 

objectionable 

deficiencies 

Very'  objectionable 
but  tolerable 
deficiencies 


For  desired  performance,  moderate  levels 
of  mental  effort  are  required 

For  adequate  performance,  moderate  levels 
of  mental  effort  are  required 

For  adequate  performance,  extensive  but 
manageable  levels  of  mental  effort  are  required 


/  Is  the  \ 

/  information  in  the  \ 
symbology  obtainable 
and  useable  within 

\a  tolerable  pilot  / 
.  workload9  / 


NO  Deficiencies 

> - ►  require 

improvements 


Major  deficiencies 

For  adequate  performance,  extensive  and  nearly 
unmanageable  mental  effort  is  required. 

7 

Major  deficiencies 

For  adequate  performance,  close-to-maximum 
and  unmanageable  mental  effort  is  required. 

8 

Major  deficiencies 

For  adequate  performance,  maximum  and 
intolerable  mental  effort  is  required. 

9 

/  Can  x 

S  information  X 
*  be  obtained  from  \ 
the  symbology  and  used 

\for  performing  / 
this  task9  / 


NO  Improvements 
- - ►  mandatory 


.  _  .  ,  c  .  .1  Symbology  cannot  be  used  for  the  performance  n 

Major  deficiencies!  0fthistas\"  10 


START 


PILOT  DECISIONS 


53 


March  2000 


P515014.PDF  [Page:  67  of  73] 

Human  Factors  Consulting 


VCS  HMD  Extension  Experiment 
Package  for  Subject  7 


Pilot  comments  and  insights 


Did  you  use  the  Symbology  Set  to  aid  performance  during  the  flight  task? 

Aircraft 

i — i — i — i _ i _ i _ i__j _ i 

Mixed  not  at  all  continually 

i _ i _ i _ i _ i  i  i  i  i 

Head 

i — i _ i — i _ i _ i _ i_j _ i 


Did  the  presentation  of  the  symbology  set  interfere  with  your  perception  of  the 
external  scene? 

Aircraft 

i _ i _ i _ i _ i _ i _ i _ i _ i 

Mixed  not  at  all  excessively 

I _ I _ I _ I _ I _ L _ I _ I _ I 

Head 


Flight  tasks/situations  where  the  various  reference  systems  would  be 
particularly  well  suited  (if  any). 


Aircraft 

Mixed 

Head 


Aspects  of  Reference  System  that  are  particularly  good. 


Aircraft 

Mixed 

Head 


Aspects  of  Reference  Systems  that  are  particularly  bad. 
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Aircraft 

Mixed 

Head 


Please  provide  any  additional  insights/comments  that  you  have  concerning 
the  symbology  set.  Are  there  further  developments  to  the  symbology  set  that  you 
would  like  to  see  take  place? 
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Reference  System  Rankings 


In  this  series  of  tests,  you  were  exposed  to  the  following  reference  systems: 

AircraftMixed  Head 

Please  rank  order  (write  in)  the  reference  systems  in  terms  of  your  preference  for  use. 
From  Most  Preferred  down  to  Least  Preferred. 


Formation 


Side  Step 
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Debrief 


•  reiterate  the  intent  of  the  experiment 

•  ask  pilots  not  to  talk  to  colleagues  about  the  specifics  of  the  experiments  (e.g.,  symbologies,  methods)  until  the 
April  timeframe. 


Secondary  interest:  the  ACD  is  a  research  and  development  tool  established  by  the  Directorate  Technical 
Airworthiness  to  assist  in  the  successful  integration  of  operator  and  machines  in  the  CF  airborne  context.  It  is 
important  to  all  agencies  that  this  capability  be  presented  to  the  user  community  through  studies  like  this  one. 


Observations 

Are  there  any  general  observations  regarding  the  facility  or  the  experiment  that  the  subjects 
would  like  to  make. 
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14.  ABSTRACT 

(U)  The  TTCP  Technical  Panel  2  (TP2)  developed  a  HUD  symbology  set  using  a  “mixed”  referencing  system  in 
which  symbols  portraying  spatial  analogue  information  are  aircraft  or  world  referenced,  whereas  non-spatial  symbols 
are  head  referenced.  One  potential  advantage  to  having  non-spatial  symbols  referenced  to  head  position  is  that 
critical  flight  and  power  information  can  be  made  available  to  the  pilot  even  when  the  pilot’s  gaze  is  directed  to  the 
side  of  the  aircraft,  such  as  during  sidestep  maneuvers.  It  is  not  clear,  however,  whether  pilots  can  effectively  use  a 
mixed  referencing  system.  For  example,  one  potential  problem  is  that  depending  on  moment-to-moment  positioning 
of  the  head,  one  or  more  of  the  head  referenced  symbols  may  overlap  with  the  aircraft  referenced  symbols.  This 
may  create  intolerable  perceptual/cognitive  confusion  and  high  mental  workload. 

There  were  two  objectives  of  the  present  research.  The  first  objects  was  to  contrast  the  effectiveness  of  the  TP2 
mixed  frames  of  reference  (FOR)  against  an  aircraft  and  a  fully  head-referenced  configuration.  To  do  this,  two 
ADS33-type  tasks  were  used:  formation  flight  and  sidestep.  The  formation  flight  results  showed  that  there  was 
undifferentiated  performance  across  the  aircraft,  mixed,  and  head  FORs.  The  sidestep  results  showed  an 
advantage  for  the  mixed  and  head  FORs  over  the  aircraft  FOR  in  two  primary  performance  measures:  maintaining 
heading  and  altitude.  The  superiority  of  the  mixed  and  head  FORs  for  the  sidestep  manuever  was  also  reflected  in 
the  pilot’s  ratings. 

A  second  objective  of  the  present  research  was  to  take  a  step  toward  establishing  a  paradigm  to  assess  the  effects 
of  space-based  and  object-based  attention  in  processing  of  HUD  symbology.  To  do  this,  pilots’  ability  to  discriminate 
targets  that  appeared  on  a  HUD  was  assessed  across  the  mixed,  aircraft,  and  head-referenced  configurations.  The 
results  of  this  experiment  were  clear  in  showing  that  the  processing  of  information  on  HUD  displays  is  affected  by 
space-based  attention.  Importantly,  the  spatial  attention  effects  occurred  in  the  head-referenced  configuration,  but 
not  in  the  aircraft  of  the  mixed  FORs.  The  comparison  of  the  head  versus  aircraft  conditions  supports  the  notion  that 
referencing  the  HUD  to  head  movements  creates  a  near-(HUD)  domain  perceptual  layer  that  is  distinct  from  the  far- 
domain  of  the  external  scene.  On  this  view,  the  effect  of  spatial  attention  occurred  with  the  head  FOR  because 
attention  is  assigned  to  the  HUD  layer,  in  a  manner  similar  to  the  placing  of  attention  on  perceptual  objects.  With  the 
aircraft  referenced  configuration,  the  HUD  (near-domain)  and  the  external  scene  (far-domain)  are  less  likely  to  form 
distinct  perceptual  layers. 
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